Hitherto, in systems for transmitting video signals of moving picture to remote place, such as, for example, television conference system or television telephone system, etc., or systems for recording video signals of moving picture onto picture recording media such as optical disc, magnetic disc or magnetic tape, etc., and reproducing recorded video signals of moving picture, in order to efficiently utilize transmission paths or recording media, video signals are caused to undergo so called efficient encoding by making use of line correlation or interframe correlation that video signals have to reduce redundancies in the spatial axis direction and in the time axis direction to transmit only significant information, thus to allow transmission efficiency to be higher.
For example, in encoding in the spatial axis direction (hereinafter referred to as intra-frame encoding), e.g., line correlation of video signal is utilized as indicated by A of FIG. 1. In the case where attempt is made to transmit respective pictures PC1, PC2, PC3 . . . constituting moving picture at times t.sub.1, t.sub.2, t.sub.3, . . . , picture data to be subjected to transmission processing is one-dimensionally encoded, e.g., within the same scanning line, or, e.g., picture is divided into a plurality of blocks to two-dimensionally encode picture data of respective blocks to thereby carry out data compression, thus to improve transmission efficiency.
Moreover, in encoding in the time axis direction (hereinafter referred to as interframe encoding), interframe correlation of video signal is utilized. Namely, as indicated by B of FIG. 1, e.g., by so called predictive encoding, picture data PC12, PC23, . . . comprised of differences (so called predictive errors) of picture data every corresponding pixels between pictures PC1 and PC2, PC2 and PC3, . . . adjacent in succession are determined to transmit these picture data PC12, PC23, . . . to thereby carry out data compression, thus to improve transmission efficiency.
Thus, as compared to the case where all of pictures PC1, PC2, PC3, . . . are transmitted, it is possible to transmit video signals by extremely lesser data quantity.
Further, in the predictive encoding in the above-described interframe encoding, in order to allow efficiency to be higher, motion-compensated predictive coding is used. Namely, in the case where, e.g., a person in the center of picture (frame) moves, etc., with respect to an object moving in picture, its motion is detected to correct position of picture data used for prediction in the previous picture (frame) by that motion to carry out predictive coding, thereby making it possible to improve coding efficiency. However, even if such a method is employed, many data must be transmitted with respect to the portion where object moves and appears from behind. In view of this, motion-compensated prediction is carried out not only in the above-described forward direction, but also in backward direction or in both directions as combination thereof, thereby permitting coding efficiency to be higher.
As the representative system of motion-compensated predictive coding as described above, so called MPEG (Moving Picture Experts Group) 2 is known. MPEG 2 is general name of the moving picture coding system which has progressed in WG (Working Group) 11 of SC (Sub Committee) 29 of JTC (Joint Technical Committee) 1 of ISO (International Standardization Organization) and IEC (International Electro-technical Commission).
In MPEG 2, Hybrid system in which motion-compensated predictive coding and Discrete Cosine Transform (hereinafter referred to as DCT) are combined is-adopted.
Sequence employed when picture is caused to undergo intra-frame coding or inter-frame coding in MPEG 2 will now be described. FIG. 2 is a view showing the configuration of so called GOP (Group of Pictures) employed in MPEG 2. In FIG. 2, there is shown the example where GOP consists of 15 frames.
Since frame F2 is caused to undergo intra-frame coding, this frame is called I picture (Intra coded picture). Since frames F5, F8, F11, F14 are predicted only from the forward direction and are caused to undergo interframe coding, those frames are called P picture (Predictive coded picture). In addition, since frames F0, F1, F3, F4, F6, F7, F9, F10, F12, F13 are predicted from the forward direction, the backward direction or both directions and are caused to undergo interframe coding, those frames are called B picture (Bidirectionally-predictive picture).
One picture is divided into N slices as shown in FIG. 3, and each slice includes M macro blocks. Each macro block includes luminance data Y0, Y1, Y2, Y3 included in four luminance blocks each comprised of 8.times.8 pixels and adjacent in upper and lower directions and in left and right directions, and color difference data Cb, Cr of color difference blocks comprised of 8.times.8 pixels in the range corresponding to these four luminance blocks. Namely, macro block includes luminance data Y1.about.Y4 of 16.times.16 pixels continuous in horizontal and vertical scanning directions with respect to luminance signal, whereas, in two color difference signals corresponding thereto, after data quantity is caused to undergo reduction processing, they are subjected to time axis multiplexing processing so that data with respect to 16.times.16 pixels of luminance signal are assigned (allocated) to respective one color difference blocks.
Arrangement of picture data within slice is such that picture data are caused to be continuous in macro block units, and picture data are caused to be continuous in block units in order of raster scan within the macro block. In MPEG 2, coding or decoding processing is carried out in the above-described macro block units.
In actual terms, moving picture encoding apparatus in conformity with the MPEG 2 includes frame memory group 12 as shown in FIG. 4, and this frame memory group 12 is supplied with picture data from pre-processing circuit (not shown) through motion vector detecting circuit 11. Namely, the pre-processing circuit separates input video signal into luminance signal and color difference signal thereafter to convert them into luminance data and color difference data which respectively consist of 8 bits, and to reduce the number of pixels of color difference data to, e.g., one fourth (1/4) in the case of, e.g., so called 4:2:0 to deliver these luminance data and color difference data to frame memory group 12 through motion vector detecting circuit 11.
The frame memory group 12 stores picture data of respective frames as picture data of reference original picture, forward original picture or backward original picture in accordance with sequence set in advance. In more practical sense, at the time of encoding picture data of frame F3 (B picture) shown in FIG. 2, for example, picture data of this frame F3 is stored as picture data of reference original picture, picture data of frame F2 (I picture) is stored as picture data of forward original picture, and picture data of frame F5 (P picture) is stored as picture data of backward original picture.
Motion vector detecting circuit 11 reads out picture data of respective frames stored in frame memory group 12 by the above-described data arrangement, i.e., block format to detect motion vector of reference original picture in macro block units. Namely, motion vector detecting circuit 11 detects motion vector of current reference picture in macro block units by using forward original picture and/or backward original picture stored in frame memory group 12. In this case, detection of motion vector is carried out so that a value such that sum of absolute values of differences between pictures of luminance data of macro block unit becomes minimum is caused to be corresponding motion vector. Then, the detected motion vector is delivered to motion compensating circuit 23, etc., and sum of absolute values of differences between pictures of macro block unit is delivered to intra-frame/forward/backward/bidirectionally predictive judging circuit 13.
This intra-frame/forward/backward/bidirectionally predictive judging circuit 13 determines predictive mode of macro block of reference original picture on the basis of this value to control predictive coding circuit 14 so as to carry out switching of intra-frame/forward/backward/bidirectional prediction in macro block units.
Predictive encoding circuit 14 is operative so that when intra-frame coding mode is set, it selects input picture data as it is, and when forward/backward/bidirectionally predictive mode is set, it selects difference (hereinafter referred to as difference data) every pixel of input picture data with respect to each predictive picture to deliver the selected data to DCT circuit 15. Namely, this predictive encoding circuit 14 includes adding circuits 14a, 14b, 14c and selector (changeover) switch 14d. These adding circuits 14a, 14b, 14c are supplied with picture data of forward predictive picture and backward predictive picture from motion compensating circuit 23 which will be described later. The adding circuit 14a generates difference data with respect to forward predictive picture, the adding circuit 14b generates difference data with respect to backward predictive picture, the adding circuit 14c generates difference data with respect to picture obtained by averaging forward predictive picture and backward predictive picture, and the selector switch 14d selects, by switching, any one of input picture data and difference data with respect to respective predictive pictures on the basis of predictive mode to deliver the selected data to DCT circuit 15.
The DCT circuit 15 allows input picture data or difference data to undergo DCT processing in block units by making use of the two-dimensional correlation of video signal to deliver coefficient data thus obtained (so called DCT coefficients) to quantizing circuit 16.
The quantizing circuit 16 quantizes coefficient data by using quantization scale (quantization step size) determined every macro block or slice to deliver quantized data thus obtained to Variable Length Coding (hereinafter referred to as VLC) circuit 17 and inverse quantizing circuit 18. In this example, quantization scale used in such quantization is determined by carrying out feedback of buffer remaining quantity of transmitting buffer memory 19 which will be described later so that it becomes equal to a value such that the transmitting buffer memory 19 dose not break down. This quantization scale is also delivered to VLC circuit 17 and inverse quantizing circuit 18.
The VLC circuit 17 implements variable length coding to quantized data along with quantization scale, predictive mode and motion vector to deliver coded data thus obtained to transmitting buffer memory 19.
The transmitting buffer memory 19 once (temporarily) stores coded data thereafter to read out it at a fixed bit rate to thereby smooth the coded data to output it as bit stream, and to feed quantization control signal of macro block unit back to quantizing circuit 16 in accordance with remaining data quantity remaining in the memory to control quantization scale. Thus, the transmitting buffer memory 19 adjusts data quantity generated as bit stream to maintain data of reasonable remaining quantity (data quantity such that overflow or underflow does not take place) in the memory. For example, when the data remaining quantity increases up to allowed upper limit, transmitting buffer memory 19 allows quantization scale of quantizing circuit 16 to be greater by quantization control signal to thereby reduce data quantity of quantized data. On the other hand, when the data remaining quantity decreases down to allowed lower limit, transmitting buffer memory 19 allows quantization scale of quantizing circuit 16 to be smaller by quantization control signal to thereby increase data quantity of quantized data.
In this way, bit streams outputted from transmitting buffer memory 19 are delivered to moving picture decoding apparatus at a fixed bit rate through transmission paths or transmission media comprised of recording medium such as optical disc, magnetic disc or magnetic tape, etc.
On the other hand, inverse quantizing circuit 18 inverse-quantizes quantized data delivered from quantizing circuit 16 to reproduce coefficient data (to which quantization distortion is added) corresponding to output of the above-described DCT circuit 15 to deliver this coefficient data to Inverse Discrete Cosine Transform (hereinafter referred to as IDCT) circuit 20.
The IDCT circuit 20 allows coefficient data to undergo IDCT processing to reproduce picture data corresponding to input picture data when intra-frame coding mode is set, and to reproduce difference data corresponding to output of predictive encoding circuit 14 when forward/backward/bidirectionally predictive mode is set to deliver them to adding circuit 21.
This adding circuit 21 is supplied, when forward/backward/bidirectionally predictive mode is set, with motion-compensated picture data of forward predictive picture or backward predictive picture (hereinafter referred to as predictive picture data) from motion compensating circuit 23, and adds this motion-compensated predictive picture data and difference data to thereby reproduce picture data corresponding to input picture data in macro block units.
Picture data reproduced in this way is stored into frame memory 22. Namely, inverse quantizing circuit 18.about.motion compensating circuit 23 constitute a local decoding circuit to locally decode quantized data outputted from quantizing circuit 16 on the basis of predictive mode to store decode picture thus obtained into frame memory 22 as forward predictive picture or backward predictive picture. In more practical sense, frame memory 22 is comprised of a semiconductor memory having memory capacity corresponding to two frames (hereinafter memory corresponding to one frame will be called frame buffer), and carries out bank switching of frame buffer to store picture data delivered from adding circuit 21 as forward predictive picture data or backward predictive picture data. With respect to the stored picture data, in dependency upon picture to be encoded, picture data of single frame is outputted as forward predictive picture data, or is outputted as backward predictive picture data. These predictive picture data are entirely the same as pictures reproduced at moving picture decoding apparatus which will be described later, and picture to be subjected to encoding processing next is caused to undergo forward/backward/bidirectionally predictive encoding on the basis of this predictive picture.
In actual terms, motion compensating circuit 23 implements motion compensation to picture data from frame memory 22 on the basis of predictive mode and motion vector to read out the motion-compensated picture data to deliver the predictive picture data thus read out to predictive coding circuit 14 and adding circuit 21 as described above.
Moving picture decoding apparatus will now be described.
The moving picture decoding apparatus includes receiving buffer memory 31 as shown in FIG. 5. Bit stream is inputted to the receiving buffer memory 31 through transmission paths or transmission media. The receiving buffer memory 31 once (temporarily) stores that bit stream thereafter to deliver it to variable length decoding circuit 32.
The variable length decoding circuit 32 allows bit stream, i.e., coded data to undergo variable length decoding to reproduce quantized data, motion vector, predictive mode, quantization scale, and the like. These quantized data and quantization scale are delivered to inverse quantizing circuit 33, and motion vector, predictive mode and the like are delivered to motion compensating circuit 38.
Operations of inverse quantizing circuit 33.about.motion compensating circuit 38 are the same as that of the local decoding circuit of the above-described moving picture encoding apparatus. These circuits carry out decoding by using quantized data, motion vector, predictive mode, quantization scale, and forward predictive picture data and backward predictive picture data already reproduced and stored in frame buffers 37a, 37b of frame memory 37. As a result, reproduction picture data is outputted from adding circuit 35 by the block format.
However, in the conventional moving picture encoding apparatus and moving picture decoding apparatus, more practical configuration for absorbing (buffering) difference of processing speed occurring between respective processing blocks was not proposed.
This invention has been made in view of such actual circumstances, and its object is to provide a picture decoding method and a picture decoding apparatus, and a picture encoding method and a picture encoding apparatus which are capable of absorbing difference of processing speed occurring between respective processing blocks.
Another object of this invention is to provide a picture decoding method and a picture decoding apparatus, and a picture encoding method and a picture encoding apparatus which are capable of improving utilization efficiency of buffer memory used for absorbing difference of processing speed between respective blocks, and reducing capacity of buffer to reduce the entire memory capacity, thus permitting the circuit scale to be reduced.